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ABSTRACT 

Inexpensive,  three-dimensional  vehicle  simulators  are  important 
visualization  tools  that  can  enhance  training  and  serve  as  low-cost 
platforms  for  testing  mobility  expert  system  algorithms.  The  moving 
vehicle  simulator  is  an  interactive,  real-time  system  that  displays  a 
dynamic,  three-dimensional,  out-the-window  view  of  the  terrain  from 
any  vehicle.  The  simulator  has  two  modes  of  operation:  stand-alone  or 
networked.  The  networked  mode  facilitates  a  missile/target  war  gam- 
ing environment.  The  simulator  can  be  easily  adapted  for  use  with  a 
variety  of  computation  resources  on  the  network. 
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1.  Introduction 

Previous  work  in  the  Graphics  and  Video  Laboratory  of  the  Department  of  Computer 
Science  at  the  Naval  Postgraduate  School  included  the  production  of  a  real-time  simulator  for 
the  Fiber  Optically  Guided  Missile  (FOG-M)  [Ref.  1,  pp.  19-27].  The  FOG-M  simulator 
displayed  a  real-time,  three  dimensional,  missile's  eye  view  of  terrain  and  vehicles  driving 
over  that  terrain.  The  FOG-M  simulator  used  digital  terrain  elevation  data  from  the  Defense 
Mapping  Agency  and  a  Silicon  Graphics,  Inc.  IRIS-3120  graphics  workstation. 

The  moving  vehicle  simulator  (VEH)  is  a  continuation  of  the  FOG-M  research  [Ref. 
2].  The  goal  of  the  follow-on  study  is  twofold.  The  first  objective  is  to  provide  a  stand-alone 
vehicle  motion  simulator.  The  second  objective  is  to  provide  more  realistic  targets  that, 
through  networking,  can  be  used  by  the  FOG-M  simulator.     One  noteworthy  aspect  of  the 


simulator  is  that  the  operator  can  display  the  out-the-windshield  view  of  any  vehicle  during 
program  execution.  The  moving  vehicle  simulator  has  been  incorporated  into  a  Mobility 
Expert  System  (MES)  and  could  easily  be  adapted  for  use  by  other  simulators  modeling  off- 
road  vehicle  motion.  It  is  the  intent  of  this  study  to  present  the  results  of  the  design,  devel- 
opment, and  implementation  of  the  moving  vehicle  simulator  and  the  networking  capabilities 
incorporated  into  the  system. 

2.  Background 

The  moving  vehicle  simulator  models  the  motion  of  remotely  piloted  vehicles,  such  as 
jeeps,  tanks,  or  trucks,  one  of  which  is  designated  the  driven  vehicle.  The  driven  vehicle 
models  a  vehicle  with  an  on-board  video  camera  capable  of  transmitting  live  pictures  of  the 
battlefield  to  a  distant  operator's  console.  The  moving  vehicle  simulator  displays  a  real- 
time, three-dimensional,  driver's  view  perspective  of  the  terrain,  and  other  vehicles.  When 
networking  is  enabled,  the  FOG-M  missile  is  also  visible.  An  interactive  user  interface  and 
a  two-dimensional  contour  map  display  allow  the  operator  to  establish  the  desired  simulator 
configuration  (stand-alone  or  networked  with  the  FOG-M  simulator)  and  to  define  each 
vehicle  to  be  used  in  the  simulation.  The  vehicle  locations,  courses,  speeds,  and  the  selec- 
tion of  a  driven  vehicle  are  determined  using  a  two-dimensional  contour  map  display. 

Once  the  simulation  begins,  a  three-dimensional  view  of  the  terrain  is  displayed.  The 
operator  can  interactively  control  the  motion  of  the  vehicle  designated  as  the  driven  vehicle. 
The  operator  controls  the  driven  vehicle's  course,  speed,  and  line-of-sight  "look"  direction 
by  the  knobs  on  a  dial  box.  The  viewing  volume  of  the  driven  vehicle  can  be  controlled  by  the 
mouse. 

3.  Terrain  Database 

Both  the  moving  vehicle  simulator  and  the  FOG-M  simulator  use  a  digital  terrain  eleva- 
tion database  provided  by  the  Defense  Mapping  Agency  (DMA)  to  draw  the  three-dimen- 
sional scene.    This  data  is  stored  as  an  array  of  sixteen  bit  data  points  that  represent  the  ter- 
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Figure       1  Terrain     Polygons 


rain  of  Fort  Hunter-Liggett,  California.  The  upper  three  bits  represent  the  height  of  the  vege- 
tation at  that  data  point.  The  lower  thirteen  bits  represent  the  terrain  elevation,  without  the 
vegetation  height  [Ref.  3,  pp.  2].  The  terrain  elevation  data  file  is  formatted  to  match  the 
two-dimensional  array  used  to  store  it  during  program  execution.  Data  points  for  ten  lengths 
of  ten  kilometers  are  stored  a  row  at  a  time,  from  west  to  east  along  a  row's  length,  and  from 
south  to  north,  going  from  row  to  row.  This  matches  the  C  compiler  storage  mapping  function 
for  two-dimensional  arrays. 


The  ten  kilometer  by  ten  kilometer  area  of  missile  flight  is  sectioned  into  one  hundred 
meter  squares,  with  each  square  consisting  of  two  triangles  (Figure  1).  The  triangles  are 
used  to  construct  a  colored,  three-dimensional  terrain  display.    Values  for  the  triangles'  coor- 
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dinates  are  determined  prior  to  missile  flight.  A  separate  program  converts  the  elevation 
height  values  from  feet  to  meters,  reads  and  scales  the  terrain  data  from  the  master  data  file, 
and  formats  the  converted  data  into  the  file  used  by  the  moving  vehicle  simulator.  Raw  ele- 
vation values  are  scaled  exponentially  by  a  factor  of  1.05  to  provide  a  more  esthetic  display. 


4.  Graphics  Hardware 

The  moving  vehicle  simulator  is  implemented  using  a  Silicon  Graphics,  Inc.  IRIS  3120 
high  performance  color  graphics  workstation.  The  workstation  uses  a  Motorola  68020  micro- 
processor and  is  available  for  under  $30,000.  The  workstation  also  uses  custom  VLSI  chips 
to  provide  hardware  clipping  and  matrix  transformations.  The  high  speed,  pipeline  architec- 
ture allows  the  performance  of  viewing,  modeling,  projection  and  display  device  rransforma- 
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Figure        3  Octant  Scan  Lines 


tions  at  a  much  greater  rate  than  would  be  possible  in  software.  The  graphics  hardware  can 
be  conceptually  depicted  as  three  pipelined  components:  the  applications/graphics  processor, 
the  geometry'  pipeline,  and  the  raster  subsystem.  The  geometry  pipeline  and  the  raster  sub- 
system are  controlled  by  the  applications/graphics  processor  [Ref.  4,  pp.  1-1].  The  IRIS  pro- 
vides a  double  buffer  display  system  with  a  resolution  of  1024  by  768  pixels. 


5.  Hidden  Surface  Elimination 

Hidden  surface  elimination  is  accomplished  by  a  real-time  implementation  of  the 
painter's  algorithm.  The  painter's  algorithm  simply  draws  objects  in  the  scene  in  depth  sort- 
ed (furthest  to  nearest)  order  [Ref.  5,  pp.  266].  For  the  terrain,  the  correct  polygon  drawing 
order  for  hidden  surface  elimination  is  an  easily  computable  function  of  the  line-of-sight  of 


the  vehicle  currently  being  operated  (Figure  2).  The  number  of  grid  squares  drawn  is  mini- 
mized by  partitioning  the  entire  viewing  circumference  into  octants.  The  drawing  order  of 
each  grid  square  within  an  octant,  from  furthest  to  nearest,  is  based  on  a  scan  line  algorithm 
(Figure  3a).  If  the  line-of-sight  is  in  the  eighth  octant,  the  scan  lines  are  defined  by  indices 
startx  and  startz.  Startz  is  incremented  until  a  stopz  is  reached.  Stopz's  are  determined  by 
the  field  of  view  calculated  from  the  viewer's  position.  Before  startz  is  incremented,  all  vehi- 
cles located  in  the  grid  square  that  was  just  drawn  are  also  drawn.  One  vertical  scan  line  is 
shown  in  (Figure  3b).  The  next  scan  line  is  drawn  by  moving  the  startx  one  position  closer  to 
the  viewer  and  repeating  the  process.  This  process  is  repeated  until  all  grid  squares  in  the 
octant  are  drawn. 

After  the  entire  scene  is  constructed,  the  vehicles  in  the  viewer's  grid  square  are  drawn 
again.  This  is  done  to  ensure  that  the  vehicles  drawn  in  adjacent  grid  squares  are  "painted 
over"  by  vehicles  in  the  viewer's  grid  square. 

Vehicles  located  in  the  center  of  a  grid  square  are  drawn  immediately  after  the  grid 
square  that  they  occupy  is  drawn.  Vehicles  crossing  grid  square  boundaries  are  drawn  only 
once.  The  grid  square  that  they  are  drawn  in  is  determined  by  the  quadrant  they  are  in  and 
the  boundary  the  vehicle  is  crossing.  A  vehicle  is  drawn  in  an  adjacent  grid  square  only  if  it 
is  near  certain  edges.  The  edges  are  determined  by  the  painter's  algorithm  and  are  shown  in 
Table  1. 


Quadrant  Grid  Square  Edge 

0  South,  West 

1  South,  East 

2  North,  East 

3  North,  West 

Table  1  Vehicle  Grid  Square  Adjustment  Table 


line-of-sight  vehicle  is  drawn  in 

in  quadrant  one .  adjacent  grid  square  if 

near  SOUTH  or  EAST  edge 


Vehicle  'B'  is  near  SOUTH  edge 
draw  it  after  grid  square  four 


Figure  4       Drawing  in  an  Adjacent  Grid  Square 


An  illustrative  example  of  the  methodology  employed  in  the  drawing  process  is  provided.  In 
Figure  4,  the  line-of-sight  from  the  driven  vehicle  'A'  is  in  quadrant  one.  With  this  line-of- 
sight,  vehicles  near  a  southern  or  eastern  grid  square  edge  are  drawn  after  the  adjacent  grid 
square  in  that  direction  rather  than  in  the  grid  square  the  vehicles  occupy.  Vehicle  'B'  in  Fig- 
ure 4  is  located  at  the  southern  edge  of  grid  square  three.  Since  the  painter's  algorithm 
draws  grid  square  three  before  grid  square  four,  the  part  of  the  vehicle  overlapping  grid 
square  four  would  be  "painted  over"  by  grid  square  four  if  the  vehicle  was  drawn  in  grid 
square  three.  To  correctly  draw  the  vehicle  and  both  grid  squares  it  overlaps,  the  vehicle 
must  be  drawn  after  grid  square  four. 
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6.  Vehicles 

In  the  moving  vehicle  simulator,  the  vehicles  are  created  as  graphical  objects.  Each 
polygon  of  each  vehicle  is  drawn  by  defining  its  vertices  and  colors,  and  then  drawing  the 
polygon  using  a  call  to  a  polygon  fill  function.  All  objects  are  created  using  backface  polygon 
removal  and  the  painter's  algorithm  to  display  an  undistorted  view  of  a  three-dimensional, 
light  shaded  object  from  any  viewing  angle  above  the  ground  plane. 

Target  vehicle  objects  (jeeps,  trucks,  tanks)  are  built  during  program  initialization. 
After  the  objects  are  constructed,  they  are  animated  and  oriented  to  the  terrain.  A  vehicle's 
course  and  speed  are  used  to  calculate  its  new  position  based  on  the  distance  it  would  have 
traveled  in  the  time  required  to  refresh  the  screen.  Each  vehicle  defined  is  associated  with 
an  element  of  one  of  three  global  two-dimensional  arrays.  There  is  one  array  for  each  of  the 
three  types  of  vehicles.  The  values  stored  in  the  arrays  are  the  integer  names  of  the  graphi- 
cal objects  to  be  drawn  in  each  terrain  grid  square.  All  vehicles  present  in  one  grid  square 
are  associated  with  the  same  element  of  the  array.  All  commands  required  to  draw  each 
type  of  vehicle  are  collected  into  the  same  graphical  object.  Vehicles  are  displayed  by  draw- 
ing the  terrain  grid  square  and  then  accessing  the  appropriate  two-dimensional  array  to  draw 
the  vehicles  that  are  present  in  that  grid  square. 

7.  Vehicle  Data  Structures 

The  moving  vehicle  simulator  uses  two  data  structures  to  manage  the  vehicle  display. 
A  linked  list  of  vehicle  definition  data  is  created  before  the  display  loop  begins  and  is  updated 
with  each  pass  through  the  loop.  Each  structure  in  the  linked  list  contains  all  the  data 
required  to  transform  and  orient  a  vehicle  object  to  the  correct  position  on  the  terrain.  One 
object  for  each  type  of  vehicle  is  created  before  the  display  loop  begins.  The  drawing  com- 
mands in  these  objects  are  used  to  draw  every  vehicle  of  that  type  used  in  the  display. 

The  second  data  structure  manages  vehicle  hidden  surface  removal.  A  single  two- 
dimensional  array  maintains  the  connection  between  the  grid  squares  and  the  order  that  the 
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Figure        5  Update  Vehicle   Grid 


vehicles  present  in  the  grid  square  must  be  drawn.  Each  element  in  the  array  contains  a  list 
of  pointers  to  records  in  the  vehicle  definition  list  for  the  vehicles  that  should  be  drawn  imme- 
diately after  drawing  the  terrain  grid  squares.  The  lists  are  maintained  in  depth  sorted  order 
(furthest  to  closest)  from  the  driven  vehicle.  The  grid  square  that  a  vehicle  should  be  drawn 
in  is  determined  by  the  vehicle's  proximity  to  a  grid  square  edge  and  the  direction  of  the  line- 
of-sight.  As  a  result,  a  vehicle  is  drawn  only  once,  regardless  of  its  position  on  the  terrain. 
As  a  vehicle  overlaps  a  grid  square,  its  position  in  the  two-dimensional  array  changes.  Fig- 
ure 5  shows  how  the  array  changes  while  maintaining  the  linked  list  depth  sorted  order.  All 
the  functions  used  to  draw  the  vehicles  and  terrain  are  performed  in  the  display  loop.    Each 


pass  through  the  loop  represents  one  frame  of  animation.     By  optimizing  the  functions,  a 
frame  rate  that  simulates  a  real-time  display  is  achieved. 

8.  Software  Implementation 

The  AT&T  Unix  System  V  operating  on  the  IRIS  workstation  supports  Fortran,  Pascal, 
Franz  Lisp,  an  optimized  C  language  compiler  and  a  complete  graphics  library.  The  C  pro- 
gramming language  was  selected  for  implementation  of  the  vehicle  simulator  to  be  compati- 
ble with  the  original  FOG-M  simulator  [Ref.  1,  pp.  19-27].  Additionally,  the  communica- 
tions packages  developed  at  the  Naval  Postgraduate  School  are  all  implemented  in  C  [Ref. 
6]. 

The  moving  vehicle  simulator  can  be  divided  into  two  operational  modes:  stand-alone 
mode  and  networked  mode.  The  stand-alone  mode  provides  an  environment  where  the  oper- 
ator can  simulate  driving  vehicles  over  the  selected  terrain.  In  the  networked  mode,  the  mov- 
ing vehicle  simulator  provides  realistic  targets  for  the  FOG-M  simulator. 

8.1  Stand- Alone  Mode 

There  are  two  fundamental  sections  of  the  stand-alone  mode:  the  initialization  phase 
and  the  vehicle  driving  simulation  phase.  The  initialization  phase  provides  an  environment 
for  vehicle  definition  and  interactive  input  of  vehicle  course,  speed,  and  position  on  the  ter- 
rain. Additionally,  the  operator  determines  the  desired  mode  (stand-alone  or  networking)  in 
this  phase.  The  driving  phase  provides  an  environment  that  dynamically  updates  the  terrain 
displays  in  real-time  based  on  operator  controlled  changes  to  the  driven  vehicle's  speed, 
course,  and  viewing  volume.  The  operator  also  designates  the  driven  vehicle. 

8.1.1       Initialization  Phase 

The  initialization  phase  is  the  interactive  input  component  of  the  moving  vehicle  simula- 
tor program.  The  display  screen  is  partitioned  into  three  areas  (Figure  6).  A  large  square 
area  (768  by  768  pixels)  on  the  left  part  of  the  screen  represents  the  two-dimensional  con- 
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tour  map  of  the  ten  kilometer  area  over  which  the  vehicles  will  operate.  The  contours  are  cre- 
ated from  the  elevation  data  in  the  DMA  digital  terrain  elevation  database.  The  map  is  color 
coded  based  on  elevation  points.  The  current  menu  is  located  in  the  upper  right  corner  of  the 
display.  Instructions  corresponding  to  the  current  menu  are  displayed  in  the  lower  right  cor- 
ner of  the  screen. 

During  this  phase,  the  operator  can  define  vehicles  by  moving  the  cursor  on  the  contour 
map  using  the  mouse.  When  the  desired  vehicle  location  on  the  map  is  selected,  the  coordi- 
nates are  locked  in  by  pressing  the  left  mouse  button.  An  icon  image  of  the  vehicle  appears 
on  the  map  at  the  specified  location.  The  operator  then  moves  the  cursor  in  the  direction  of 
the  desired  vehicle  course.  A  rubberband  line,  originating  at  the  icon  image,  shows  the 
potential  vehicle  course.  Pressing  the  left  mouse  button  locks  in  the  course  represented  by 
the  direction  of  the  rubberband  line  from  the  vehicle's  defined  location.  A  slider  speedometer 
appears  in  the  menu  area  to  allow  the  operator  to  set  the  vehicle's  speed  by  moving  the  cur- 
sor and  pressing  the  left  mouse  button.  Once  all  desired  vehicles  have  been  defined,  the 
actual  simulation  can  begin. 

8.1.2        Vehicle  Driving  Simulation 

The  driving  simulation  phase  provides  successive  real-time  terrain  displays  to  the  oper- 
ator as  the  vehicle  moves  over  the  terrain.  The  simulation  begins  with  the  designation  of  a 
driven  vehicle  selected  from  the  previously  defined  vehicles.  The  driven  vehicle  is  selected 
by  moving  the  cursor  over  the  vehicle's  icon  image  on  the  map  and  then  depressing  the  left 
mouse  button.  Selection  of  a  vehicle  starts  the  display  loop  of  the  simulation.  In  networked 
mode,  the  vehicle  simulator  waits  until  the  missile  launch  occurs  before  entering  the  display 
loop. 

The  driving  display  is  partitioned  into  four  areas  (Figure  7).  The  large  square  area  to 
the  left  (768  by  768  pixels)  represents  the  out-the-window  view  as  seen  from  the  driven 
vehicle.    An  operating  menu  is  displayed  in  the  upper  right  side  of  the  screen  that  allows  the 
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operator  to  change  vehicles  or  terminate  the  program.  A  contour  map  with  the  position  of  the 
driven  vehicle  and  its  viewing  volume  is  displayed  on  the  right,  center  section  of  the  screen. 
The  driven  vehicle's  speed,  view  direction  and  available  operator  controls  are  shown  in  the 
lower  right  section  of  the  screen. 

8.2  Networked  Mode 

The  moving  vehicle  simulator  is  the  first  attempt  at  the  Naval  Postgraduate  School  to 
produce  a  network  of  real-time,  interactive  moving  platform  simulators.  The  network  commu- 
nication protocol  selected  is  normal  (blocking)  socket  I/O  [Ref.  6].  Blocking  I/O  allows  syn- 
chronous operation  of  the  FOG-M  and  moving  vehicle  simulators.  A  pair  of  sockets  is  used 
to  transfer  and  guarantee  delivery  of  the  socket  stream  data  between  the  two  simulators. 
The  moving  vehicle  simulator  acts  as  the  server  to  the  client  FOG-M  simulator. 

Operating  the  moving  vehicle  simulator  in  conjunction  with  the  FOG-M  simulator 
requires  establishing  network  data  paths.  This  is  accomplished  through  the  creation  of  dedi- 
cated sockets  for  read  and  write  paths  for  both  control  and  data.  Failure  to  establish  the 
communications  paths  causes  the  simulators  to  default  to  the  stand-alone  mode  of  operation. 

Prior  to  missile  launch,  the  missile  operator's  console  is  provided  with  relevant  vehicle 
information;  i.e.,  the  number  and  types  of  vehicles  defined.  Handshaking  takes  place  after  ini- 
tial data  transfer  and  before  entering  the  display  loop  to  allow  either  console  to  abort  the  sim- 
ulation. If  either  simulation  is  aborted,  the  other  can  continue  in  stand-alone  mode.  After 
completion  of  the  initial  set-up,  the  FOG-M  simulation  console  waits  for  the  vehicle  defini- 
tion data  from  the  moving  vehicle  simulator  before  allowing  missile  launch.  The  moving  vehi- 
cle simulation  waits  for  the  launch  event  before  entering  the  display  loop  to  insure  simulator 
synchronization.  Regardless  of  the  number  of  vehicles  in  the  missile  flight  area,  only  the 
driven  vehicle's  information  is  sent  to  the  missile  console.  The  position  of  the  other  vehicles 
is  predicted  based  on  their  initial  position,  course  and  speed. 
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The  missile  simulator  transfers  a  status  flag  to  the  moving  vehicle  simulator  indicating  if 
the  missile  is  still  in  flight.  If  the  missile  is  still  flying,  it  sends  missile  position  and  course 
data.  If  it  is  no  longer  flying,  it  sends  the  identity  of  the  vehicle  destroyed. 

9.  System  Features  and  Limitations 

Currently,  the  system  allows  only  one  console  of  each  simulator  type  in  a  dedicated  link 
arrangement  to  be  networked  together.  To  insure  synchronization,  a  console  can  not  proceed 
past  a  socket  read  until  the  information  is  obtained.  This  lock-step  execution  prevents  the 
vehicle  console  operator  from  changing  the  driven  vehicle  while  the  missile  is  in  flight. 

As  in  the  FOG-M  simulator,  the  inability  to  display  the  terrain  using  the  12.5  meter 
samples  available  in  the  terrain  data  file  is  another  limitation  of  this  simulator.  To  utilize  the 
12.5  meter  samples  would  require  some  64  times  more  triangles  in  the  terrain  display.  The 
simulator  uses  100  meter  samples  and  that  reduces  the  degree  of  variation  in  the  terrain  sur- 
face. The  use  of  100  meter  samples  is  all  the  IRIS-3120  can  currently  support  for  real-time 
display. 

System  performance  for  the  networked  mode,  stand-alone  mode,  and  the  mobility 
expert  system  (MES)  are  shown  in  Table  2. 


Simulator  Mode 

Number  of  Vehicles 

Frames  Per  Second 

Networked 

1  (static) 

2.6 

10  (static) 

1.9 

1  (dynamic) 

1.4 

10  (dynamic) 

1.2 

Stand-Alone 

1  (static) 

5.7 

10  (static) 

4.0 

1  (dynamic) 

5.3 

10  (dynamic) 

4.3 

MES 

1  (dynamic) 

3.7 

10  (dynamic) 

3.3 

Table  2  Display  Update  Rates 
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Figure    8       Program    Hierarchy 


Static  refers  to  the  type  of  vehicle  objects  drawn  in  the  original  FOG-M  simulator.  Dynamic 
refers  to  vehicle  objects  that  more  closely  reflect  normal  vehicle  dynamics  over  natural  ter- 
rain. The  vehicle  dynamics  modeled  in  the  MES  are  more  complicated  than  the  dynamics 
modeled  in  the  other  simulators,  resulting  in  a  slower  frame  update  rate. 

10.  VEH  as  a  Visualization  Tool  for  a  Mobility  Expert  System 

Above,  we  describe  the  moving  vehicle  simulator  as  either  stand-alone  or  as  a  net- 
worked player  to  the  FOG-M  simulator.  It  is  actually  an  important  visualization  tool. 
Research  is  ongoing  to  develop  new  applications  around  the  moving  vehicle  simulator.  An 
enhanced  version  of  the  moving  vehicle  simulator  is  being  used  in  conjunction  with  a  Mobility 
Expert  System  (MES)  currently  under  development  at  the  Naval  Postgraduate  School. 
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10.1        MES  Goals  and  Objectives 

The  development  of  expert  system  based  coordination  algorithms  for  groups  of 
autonomous  vehicles  is  the  major  objective  of  the  MES  project.  The  second  objective  is  to 
develop  the  software  necessary  to  create  motion  simulation  of  the  system  using  realistic 
vehicle  dynamics  over  a  computer  generated  terrain  model.  For  purposes  of  this  study,  the 
prototype  system  developed  closely  follows  the  model  of  the  FMC  autopilot  [Ref.  7].  The 
program  hierarchy  is  shown  in  Figure  8. 

The  MES  is  a  system  using  four  different  computer  architectures,  three  programming 
languages,  four  networking  packages,  three  operating  systems  and  an  expert  system  shell. 
The  four  computer  architectures  used  are:  the  Symbolics  3600  line  of  LISP  Machines,  the 
Texas  Instrument  Explorer  LISP  Machine,  the  Silicon  Graphics,  Inc.  IRIS  Graphics  Worksta- 
tion, and  the  Digital  Equipment  Corporation  VAX  11/785.  The  operating  systems  utilized  in 
the  project  are  the  Unix  operating  system  (4.3BSD  and  ATT  System  V.3),  the  Symbolics 
Genera  system  and  the  Texas  Instruments  Explorer  system.  The  languages  implementing 
the  system  are  Prolog,  C,  and  Lisp  with  flavor  extensions.  The  expert  system  shell  used  is 
the  KEE  expert  system. 

A  high  level  Command  and  Control  Subsystem  (CCS)  is  simulated  on  a  LISP  Machine 
and  a  VAX.  The  CCS  provides  centralized  autonomous  command  and  control  functions  to 
the  individual  tanks  and  acts  as  a  single  interface  to  the  autonomous  vehicles  in  the  unit. 
This  allows  an  isolation  of  observable  phenomena  for  the  tactical  assessment  function  as 
well  as  centralizing  the  focus  of  one  problem  in  the  research  area. 

Simulated  tanks  with  the  characteristics  of  the  existing  FMC  Autonomous  Land  Vehi- 
cle are  modeled  as  in  [Ref.  7].  The  model  is  conceptually  organized  into  two  distinct  parts: 
(1)  the  graphics  instantiation,  with  vehicle  controller  functions  on  the  IRIS,  and  (2)  the  rule- 
based,  expert  system  behavior,  implemented  on  the  Lisp  Machines.  The  tanks  operate 
autonomously  in  much  the  same  way  as  the  FMC  vehicle  [Ref.  7].  Specifically,  each  tank 
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possesses  a  simulated  vision  capability,  an  autopilot,  and  the  ability  to  send  vehicle  steering 
and  reference  velocity  commands  to  a  vehicle  controller. 

10.2        Autonomous  Tank  Rules 

Individual  tanks  perform  according  to  the  algorithm  presented  in  Figure  9.  The  autopilot 
possesses  capabilities  in  addition  to  those  being  developed  at  FMC  [Ref.  8].  These  extra 
capabilities  allow  the  vehicle  to  act  as  an  integral  part  of  a  tactical  autonomous  unit.  A 
tank's  designated  place  in  a  tactical  formation  is  based  on  the  commands  sent  to  it  from  the 
lead  tank.  The  tank  maintains  its  station  in  the  formation  until  it  receives  new  commands. 
Currently  the  tanks  use  three  sets  of  simple  rules  that  allow  the  vehicles  to  assume  a  line, 
column,  or  file  formation  [Ref.  9].  For  each  formation,  each  tank  possesses  knowledge  about 
who  it  is,  the  type  of  formation,  its  guide  vehicle,  and  the  vehicles  that  should  be  to  its  flanks, 
front  and  rear.  Rules  for  each  formation  are  divided  into  four  functional  categories:  collision 
avoidance,  speed  determination,  direction  determination,  and  stationing.  These  rules  are  pre- 
sented in  Figures  10  through  13. 

An  autonomous  tank  is  comprised  of  a  set  of  functions  that  reside  on  a  LISP  machine. 
The  autonomous  tank's  controller  and  graphics  object  reside  on  the  IRIS.  Each  LISP 
machine  controls  a  graphically  rendered  tank  on  the  IRIS  battlefield  during  a  simulation  run. 
The  Lisp  functions  perform  the  algorithms  presented  in  Figures  9  through  13.  Each  LISP 
machine  generates  task  commands  that  are  sent  to  the  individual  tank  that  it  controls.  The 
LISP  machines  also  determine  the  approximate  time  interval  required  for  the  tank  to  respond 
to  the  task  command. 

The  tanks  perform  a  simulated  visual  scan  of  the  environment  in  the  IRIS  and  produce 
high-level  observations  about  the  battlefield.  These  observations  are  used  to  perform  tacti- 
cal assessments  and  create  tasks  to  accomplish  goals  using  rule-based  inference  engines.  A 
rule-based  inference  engine  is  a  program  that  processes  if<circumstances>then<do-task> 
type  expressions.  These  expressions  are  constructed  through  the  interrogation  of  an  expert. 
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Loop 

Check  for  commands  from  the  Command  and  Control  Subsystem. 
If  change  in  formation,  acquire  rules  and  facts 
necessary  from  disk  storage  and  implement 

Perform  a  visual  scan  of  the  environment 

For  each  object  identified: 

Establish  its  position  in  reference  to  the 
tank's  body  coordinate  system. 

Approximate  its  future  location  at  beginning 
of  next  iteration  of  the  algorithm. 

Produce  low  level  observations  about  the 
object  as  input  to  the  task  generator. 

EndFor 

Generate  tasks  in  the  task  generator  using  the  low  level 
observations  and  knowledge  and  rules  necessary  to 
complete  currenUy  assigned  goals. 

Display  diagnostic  information  and  explanations  for  each 
task  generated. 

Execute  communications  tasks  to  Command  and  Control 
subsystem. 

Execute  tasks  generated  by  communicating  sequences 
of  vehicle  steer  and  reference  velocity  commands 
to  the  vehicle  controller  residing  on  the  IRIS. 

EndLoop. 

Figure  9  Autonomous  Tank  Control  Algorithm 
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Avoid  Collision  To  The  Right: 
If 

the  vehicle  is  or  will  be  too  close  to  an  object,  and 
the  object  is  to  the  right  of  the  vehicle, 
Then 
move  to  the  left. 

Avoid  Collision  To  The  Left: 

If 
the  vehicle  is  or  will  be  too  close  to  an  object,  and 
the  object  is  to  the  left  of  the  vehicle, 

Then 
move  to  the  right. 

Avoid  Collision  Ahead: 
If 
the  vehicle  is  or  will  be  too  close  to  an  object,  and 
the  object  is  ahead  of  the  vehicle, 
Then 
If 
not  enough  time  to  maneuver, 
Then 
Stop. 
Elself 

able  to  maneuver, 
Then 
maneuver  around  object  in  flank 
with  greatest  maneuvering  room. 

Avoid  Collision  From  Behind: 

If 

the  vehicle  is  or  will  be  too  close  to  an  object,  and 
the  object  is  behind  the  vehicle  and  closing, 

Then 
match  the  object's  speed. 


Figure  10  Collision  Avoidance  Rules 
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Change  Speed: 
If 

vehicle  is  on  course  with  its  guide  vehicle,  and 

vehicle  is  behind  or  ahead  of  its  station, 
Then 

change  speed  to  move  vehicle  to  position  by 

next  iteration  of  tank  algorithm. 

Match  Speed: 

If 
vehicle  is  on  course  with  its  guide  vehicle,  and 
vehicle  is  on  station  with  its  guide  vehicle, 

Then 
match  speed  of  the  guide  vehicle. 

Stop: 

If 
guide  vehicle  is  stopped,  and 
vehicle  on  station  with  guide  vehicle, 

Then 
stop  vehicle  on  station. 


Figure  1 1  Speed  Determination  Rules 


Turn  Left: 
If 
vehicle  is  off  course  from  its  guide  vehicle  and 
relative  right  to  the  direction  of  guide  vehicle's 
course, 
Then 
turn  left  the  angular  difference  to  come  about. 

Turn  Right: 
If 

vehicle  is  off  course  from  its  guide  vehicle  and 

relative  left  to  the  direction  of  guide  vehicle's 

course, 
Then 

turn  right  the  angular  difference  to  come  about. 


Figure  12  Direction  Determination  Rules 
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Close  Right  With  Guide: 
If 

vehicle  is  too  far  from  guide,  and 
vehicle  is  left  of  guide,  and 
guide  vehicle  is  normally  vehicle's  right  vehicle, 
Then 
move  to  the  right. 

Close  Left  With  Guide 
If 

vehicle  is  too  far  from  guide,  and 

vehicle  is  right  of  guide,  and 

guide  vehicle  is  normally  vehicle's  left  vehicle, 
Then 

move  to  the  left. 

Assume  Correct  Position  in  Relation  to  Guide: 
If 

vehicle  is  on  course  with  guide,  and 

vehicle  is  left/right  of  guide, 

but  vehicle  should  be  right/left  of  guide, 
Then 

drop  behind  guide, 

turn  90  degrees  right/left, 

proceed  until  past  guide, 

turn  90  degrees  left/right 


Figure  13  Stationing  Rules 
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>  (gettanks  "lineformation") 
T 

>  (start-the-battle  1  3) 

Tank  #1  now  conscious 

name    =  1 

name  of  the  tank  that  is  conscious 

xl       =  5300.49 

grid  coordinates  of  tank  1 

zl      =  1743.12 

speed   =0.0 

tank  1  is  not  moving 

direct  =  302.25 

tank  l's course 

r-angle  =  -57.75 

tank  1  's  course  relative  to  compass  north 

name    =2 

name  of  tank  being  compared  to  tank  1 

x2      =  5408.97 

grid  coordinates  of  tank  2 

z2      =  1720.71 

speed   =  1.0 

speed  of  tank  2 

direct  =303.10 

tank  2's  course 

reldir  =  0.0 

tank  2's  course  relative  to  tank  1  's  course 

x2rel   =38.93 

tank  2's  position  relative  to 

z2rel   =-103.70 

tank  l's  position 

x2nxt   =38.93 

estimated  position  of  tank  2  when 

z2nxt   =-84.37 

tank  1  becomes  conscious  again 

distance^  19.33 

rel  distance  between  tank  1  and  2 

when  tank  1  becomes  conscious  again 

(14a) 

name    =  1 

xl      =5300.49 

zl      =  1743.12 

speed  =0.0 

direct  =  302.25 

r-angle  =  -57.75 

name    =  3 

x3      =5432.12 

z3      =  1809.47 

speed   =  1.0 

direct  =  302.25 

reldir  =0.0 

x3rel   =126.35 

z3rel   =-75.92 

x3nxt   =  126.35 

z3nxt  =-56.58 

distance=  19.33 

(14b) 

Figure  14  J 

Jingle  Iteration  of  Start-the-Battle 
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(RULE  CLOSE-RIGHT  SAYS  TASK  MOVE-TO-RIGHT  1) 
(RULE  DIRECTIONS  SAYS  LEFT  IS  OPPOSITE  OF  RIGHT) 
(RULE  DIRECTIONS  SAYS  RIGHT  IS  OPPOSITE  OF  LEFT) 
(RULE  STOP  SAYS  TASK  STOP  1) 

(14c) 


(TASK  MOVE-TO-RIGHT  1  BECAUSE) 

(RIGHT  VEHICLE  IS  2) 

(1  IS  LEFT  OF  2) 

(1  WILL  BE  LEFT  OF  2) 

(1  WILL  BE  TOO  FAR  FROM  2) 

(GUIDE  VEHICLE  IS  2) 

(VEHICLE  IS  1) 

(FORMATION  IS  LINE) 


(14d) 


(TASK  STOP  1  BECAUSE) 
(1  WILL  BE  AHEAD  OF  2) 
(GUIDE  VEHICLE  IS  2) 
(VEHICLE  IS  1) 
(FORMATION  IS  LINE) 


(14e) 


Figure  14  Single  Iteration  of  Start- the-Battle  (continued) 
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Tank  #1  now  conscious 


name    =  1 
xl       =4474.45 
zl      =2411.60 
speed   =  1.46 
direct  =  302.25 
r-angle  =  57.75 

name    =3 
x3      =  4453.29 
z3      =2428.31 
speed   =  1.0 
direct  =  302.25 

reldir  =  0.0 

x3rel  =2.83 
z3rel   =26.81 

x3nxt  =2.83 
z3nxt  =7.17 
distance^  19.63 


(RULE  AVOID-COLLISION-TO-RIGHT  SAYS  TASK  MOVE-TO-LEFT  1) 
(RULE  DIRECTIONS  SAYS  LEFT  IS  OPPOSITE  OF  RIGHT) 
(RULE  DIRECTIONS  SAYS  RIGHT  IS  OPPOSITE  OF  LEFT) 


(TASK  MOVE-TO-LEFT  1  BECAUSE) 

(1  WILL  BE  LEFT  OF  3) 

(1  WILL  BE  TOO  CLOSE  TO  3) 

(VEHICLE  IS  1) 

(FORMATION  IS  LINE) 


Figure  15  Reasoning  about  Future  Events 
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Typical  tasks,  such  as  those  generated  for  formation  keeping,  are  vehicle  referent  velocities 
and  directions.  These  tasks  are  transmitted  to  the  vehicle  controller  residing  on  the  IRIS. 
The  vehicle  controller  then  executes  the  tasks  and  communicates  feedback  information  to  the 
requesting  Lisp  Machine. 

10.3        A  Single  Iteration  of  Start-the-Battle 

Figure  14  presents  a  single  iteration  of  the  tank  algorithm  for  tank  1  operating  in  con- 
junction with  two  other  vehicles,  tank  2  and  tank  3.  The  information  in  Figure  14  is  taken 
from  the  display  of  the  Lisp  Machine  designated  as  tank  1.  Figure  14a  shows  both  tank  1 
and  tank  2's  grid  coordinates,  course,  speed,  and  information  about  tank  2's  position,  course, 
and  speed  relative  to  tank  1.  Figure  14b  shows  the  same  information  for  tank  3.  Figure  14c 
shows  the  rules  needed  to  move  a  tank  to  the  right.  Figure  14d  shows  the  rules  used  for  a 
line  formation  for  maintaining  a  separation  interval  between  two  tanks.  Figure  14e  shows 
the  rule  used  when  tank  2  is  the  guide  tank  and  tank  1  is  too  far  ahead  of  the  guide  tank.  As 
a  result,  tank  1  is  ordered  to  stop.  Once  the  guide  tank  catches  up,  another  set  of  rules  (not 
shown)  is  used  to  order  tank  1  to  increase  speed. 

The  tanks  reason  about  the  IRIS  battlefield  world  relative  to  their  own  individual  body 
coordinate  systems.  The  tanks  reason  about  time  by  approximating  positions,  dispositions, 
and  possible  intentions  of  objects  in  view  during  possible  future  event  time  frames.  Tanks 
also  continuously  re-evaluate  their  individual  circumstances  as  well  as  their  vehicle  con- 
troller's response  time  to  a  direction  or  velocity  command.  This  allows  a  tank  to  predict  and 
address  future  events.  Figure  15  provides  an  example. 

In  Figure  15,  x3rel  and  z3rel  are  the  x  and  z  coordinates  of  tank  3  relative  to  tank  1  at 
time  t.  The  variable  x3nxt  and  z3nxt  are  the  predicted  x  and  z  coordinates  of  tank  3  relative 
to  tank  l's  predicted  future  location  at  time  t' '.  Tank  1  will  be  too  close  to  tank  3  because  the 
horizontal  interval  distance  will  exceed  the  value  of  a  constant  measure  called  proper-inter- 
val as  tank  1  approaches  tank  3  from  behind.    The  proper  interval  is  the  required  distance 
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Figure  16  Moving  to  the  Line  of  Departure 


Figure  17    Crossing  the  Line  of  Departure 
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between  two  tanks  in  the  formation.  This  distance  varies  depending  on  the  type  of  formation 
being  executed.  When  the  distance  between  the  two  tanks  is  less  than  the  proper  interval,  a 
task  is  generated  by  tank  1  to  increase  the  distance  between  the  two  tanks. 

10.4        A  Typical  Test  Mission 

Figures  16  through  19  illustrate  a  typical  test  mission.  Figure  16  depicts  the  movement 
from  an  assembly  area.  The  initialization  phase  for  the  IRIS  has  been  conducted,  the  tactical 
assessment  carried  out,  and  four  LISP  machines  have  been  initialized  to  drive  four  of  the 
tanks  in  the  unit.  The  guide  vehicle  for  the  unit,  driven  by  a  human  operator  on  the  IRIS,  has 
been  given  an  initial  direction  and  speed.  The  jeep  was  then  selected  to  view  the  formation 
as  it  turned  to  its  left  to  assume  a  column  formation.  The  picture  was  taken  from  the  jeep. 

Figure  17  depicts  the  column  after  crossing  the  line  of  departure  and  conducting  move- 
ment to  contact  (going  out  and  engaging  the  enemy).  The  guide  vehicle  is  the  lead  tank  in 
the  column.  To  obtain  the  picture,  the  jeep  was  driven  to  a  known  destination  of  the  lead 
tank.  The  jeep  then  was  positioned  to  get  a  view  as  the  column  approached. 

Figure  18  depicts  the  actions  at  the  final  coordination  line.  The  unit  deployed  into  a  line 
formation  and  is  about  to  move  through  the  objective.  This  deployment  was  effected  with  the 
help  of  manual  intervention.  The  guide  tank  was  stopped  at  the  final  coordination  line  by  a 
human  operator.  This  forced  the  column  to  halt  by  initiating  certain  station  keeping  rules. 
The  function  application  of  Start-the-battle  was  allowed  to  expire  upon  each  LISP  machine. 
A  new  formation  was  then  acquired  by  each  LISP  machine.  The  function  Start-the-battle  was 
then  re-applied  upon  each  LISP  machine.  The  human  operator  assumed  control  of  the  guide 
vehicle  while  the  autonomous,  LISP  machine  driven  tanks  then  assumed  their  positions  in 
the  line  formation  after  about  30  seconds  of  maneuvering. 
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Figure  19  depicts  a  flanking  view  of  the  line  of  tanks  as  they  assault  an  objective.  The 
line  is  sweeping  past  the  stationary  jeep  from  which  the  picture  was  taken.  The  fifth  tank  in 
the  line  decided  to  maneuver  around  the  other  side  of  the  jeep  and  is  not  depicted. 

10.5        MES  Implementation 

The  MES  system  is  distributed  across  the  various  specialized  architectures  in  accor- 
dance with  hardware  capabilities.  Thus,  it  was  possible  to  create  an  entirely  satisfactory 
real-time  system  at  low  cost.  The  current  suite  of  equipment  allows  up  to  five  individual 
tanks  to  operate  on  the  battlefield  represented  on  the  IRIS. 

Performance  bottlenecks  occur  during  communication  processing  on  the  IRIS.  This  is 
because  each  tank  spawns  a  send  and  receive  process  to  communicate  to  a  Lisp  machine. 
The  performance  bottlenecks  on  the  Lisp  machine  side  are  in  relation  to  the  sequential  nature 
of  the  command  and  control  system's  execution.  The  problem  is  that  the  vision  and  inference 
operations  are  not  concurrent  or  continuous.  The  LISP  machine  must  ask  the  IRIS  for  vision 
information  and  then  wait  until  the  IRIS  collects  and  returns  the  vision  information.  Once  it 
has  the  information,  it  uses  the  information  to  make  inferences  about  the  tank  it  is  controlling 
relative  to  the  other  tanks  on  the  battlefield. 

11.  Conclusion 

We  have  described  how  one  extends  the  capabilities  of  inexpensive  three-dimensional 
visual  simulators  on  individual  workstations  to  the  networked  workstation  environment. 
Individual  graphics  workstations  are  easily  grown  out  of  as  our  applications  become  more 
sophisticated.  We  grow  out  onto  a  network  of  workstations  to  allow  for  other  players  or  to 
partition  our  system  into  processes  computed  by  separate  machines.  For  the  system 
described  above,  the  partitioning  across  machine  boundaries  has  been  expensive  due  to  the 
lack  of  readily  available  networked  graphics  and  computing  software  facilities.  We  expect 
this  to  change  as  high-performance  graphics  workstation  manufacturers  recognize  the  impor- 
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tance  of  distributed  graphics  and  computational  functionality.  Ideas  such  as  location  indepen- 
dent computing  and  location  independent  graphical  objects  are  a  step  in  the  right  direction. 

We  have  also  shown  how  the  notion  of  inexpensive  three-dimensional  visual  simula- 
tors as  visualization  tools  can  lead  to  better  understanding  for  typically  graphics-less  areas 
such  as  expert  systems.  Three-dimensional  simulators  that  can  be  readily  "plugged-into" 
diverse  computational  environments  present  a  viable  alternative  for  the  future.  To  accom- 
plish this  goal,  we  need  to  make  our  three-dimensional  visual  simulators  inexpensive  and 
adaptable. 
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